Augmented reality systems project virtual characters and objects into physical locations, allowing for immersive experiences and novel interaction models. Typically, an augmented reality device calculates a position of a virtual object while capturing video images of the device environment (e.g., the environment of a room in which the user of the augmented reality device is currently located). When the position associated with the virtual object is visible in the captured images, the augmented reality device may render the virtual character, for example by overlaying the environment video with a two- or three-dimensional animated rendering of the virtual character.